fix: concatenate filepattern alias contents once per alias instead of once per flag#604
Merged
cspath1 merged 3 commits intoJul 15, 2026
Merged
Conversation
GenerateAliases rebuilt the full concatenation of matched file contents (plus a []byte-to-string copy) for every flag key. On a repo with ~80MB of matched source and ~160 flags, peak allocation reached tens of GB and the process was OOM-killed. Hoist the concatenation to GenerateAliases and reuse it across flags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cspath1
approved these changes
Jul 15, 2026
cspath1
left a comment
cspath1
left a comment
Contributor
There was a problem hiding this comment.
Thank you for this! It looks like there's a minor lint issue with allFileContents being unused now. Approving now so it can be merged once that's been fixed!
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
cspath1
approved these changes
Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Disclaimer: I used AI to generate this PR and fix - if that's not allowed please let me know.
Problem
GenerateAliasescallsGenerateAliasesFromFilePatternonce per flag key, and that function rebuilds the full concatenation of all glob-matched file contents (plus a[]byte→stringcopy for the regex engine) on every call.On a repo with ~80MB of matched source and ~160 flags, peak allocation reaches tens of GB and the process is OOM-killed (exit 137) before producing any output. The same blowup affects find-code-references-in-pull-request, which consumes this package as a library and passes all flag keys through the same path.
Fix
Hoist the concatenation out of the per-flag loop:
GenerateAliasesnow builds each filepattern alias's contents once and reuses it across flags.GenerateAliasesFromFilePatternkeeps its public signature and behavior, delegating to the extractedconcatFilePatternContents+matchFilePatternAliaseshelpers.With this change the same scan (~160 flags, ~80MB matched contents, alpine container) completes in ~14s with no OOM.
Testing
Existing suite passes unchanged; no behavior change, output is identical for the repos we compared before/after.
Prior art
#491 (for #377, "Tool gets stuck when defining glob filepattern on big repo") added glob and regex caching on this same path, which addressed the CPU side of big-repo filepattern scans. This change addresses the remaining memory side: the per-flag rebuild of the concatenated file contents.